Fusion ARTMAP: An Adaptive Fuzzy Network for Multi-Channel Classification

Fusion ARTMAP is a selforganizing neural network architecture for multiMchannel, or InultiMscnsor, data fusion. Fusion ARTMAP generalizes the fuzzy ARTMAP architecture in order to adaptively classify multi-channel data. The network has a sy1nmetric organization such that each channel can be dynamically configured to serve as either a data input or a teaching input to the system. An ART module forms a compressed recognition code within each channel. These codes, in turn, beco1ne inputs to a single ART system that organizes the global recognition code. When a predictive error occurs, a process called parallel mateh tracking sin1ultaneously raises vigH ilanees in tnultiple ART modules until reset is triggered in o1w of thmn. Parallel tnateh tracking hereby resets only that portion of the recognition code with the poorest tnatch, or minitnu1n predietive eonfidenee. This internally controlled selective reset process is a type of credit assignn1ent that creates a parsimoniously connected learned network. I. MULTI-CHANNEL DATA FUSION Fusion ARTMAP is a neural network architcc*Supported in part by ARPA (ONR N00014N92-J-401J), the National Science Foundation (NSF IRI 90-00530), and the Office of Naval Research (ONR N00014-91-J-4100). fSupportedin part by British Petroleum (BP 89-A-1204), ARPA (ONR N00014-92-J-4015), the National Science Foundation (NSF IRI-90-00530), and the Office of Naval Research (ONR N00014-91-J-4100). tSupported in part by ARPA (ONR N00014-92-J-401J), the National Science Foundation (NSF IRI-90-24877), and the Office of Naval Research (ONR N00014-91-J-4100). §Supported in part by the Air Force Office of Scientific Research (AFOSR F49620-92-J-0334), a National Science Foundation Graduate Fellowship, and the Office of Naval Re,ea.·ch (ONr\ N0014-91-J-4100). 1 ture designed to adaptively classify objects using multiple sources of information, regardless of its source or type. An example of the fusion problem is the cht.ssification of trucks based on inputs from different types of sensors such as range, doppler, and camera. Alternatively, multiple input sources could represent different views of the truck, such as top, front, and side views. Trucks can also be classified using different spatial scales by combining information from cameras that zoom in on the tires and information from cameras that provide a view of t.he whole truck. In general, Fusion ARTMAP is designed to classify objects using information from multiple sources of any type. One straightforward approach to the fusion problem is vector concatenation. That is, inputs from each channel are joined to form one large vector that then becomes the input to a single-channel supervised learning system. This approach is used, for example, by Chu and Aggarwal [7] to train a back-propagation neural network on inputs from infrared, range, and visual sensors. Vv'henever the classifier makes a wrong prediction during training, it is desirable to modify some system parameters in order to improve the total system performance. Deciding which parameters to modify is known as the the credit assignment problem. Since the information from the different sensors is concatenated into a single feature vector, the predictive power of each individual sensor is unknown to the classifier. Therefore, the credit assignment problem is solved by assigning blame nonspecifically to all input channels. Failure to account for the individual channels' predictive power leads to connectivity that tends to grow multiplicatively with the size of the input vector. Fusion A IlTMAP utilizes a modular approach to sensor fusion. Each sensor is assigned an individual classifier, the outputs of which serve as the inputs to a global classifier which makes a global prediction. For example, information from a range sensor is first classified into depth codes while information from a doppler sensor is classified into speed codes. The compressed depth and speed codes become inputs to a global classifier, which predicts the type of truck. By assigning an individual classifier to each sensor channel, blame can be assigned selectively to the channels with lowest predictive confidence. Such an approach retains system predictive accuracy while reducing total network connectivity by maximizing compression within each channel. Fusion ARTMAP uses the multi-channel structure of the input data to streamline the network design. One intra-channel code can contribute to several global codes, leading to reduced network connectivity. In addition, teacher and data input channels are dynamically defined via gain control, so each channel can play either the role of a teacher or the role of an input at different times. Gain control also allows the system to function correctly even if input data to certain channels is missing at various times. Thus, faulty sensors may be deleted or new sensors added as the need arises. II. FUZZY ARTMAP: A FUSION BUILDING BLOCK Fuzzy AHTMAP is a supervised neural network classifier that learns to classify inputs by a fu~zy set of features, or a pattern of fuz;zy rnembership values between 0 and 1 indicating the extent to which each feature is present. Fuzzy AHTMAP differs from many other fuzzy pattern recognition algorithms [2],[9] in that it learns each input as it. is received on-line, rather than by performing an off-line optimization of a criterion function. Each fuzzy AHTMAP system consists of a pair of fuzzy AHT classifiers (AHT,, and ARTb) that. create stable recognition categories in response to arbitrary sequences of input patterns (Fig. 1). During supervised learning, ARTa receives a stream a(P) of input patterns, and AR'I'b receives a stream b(P) of input patterns, where h(ll) is the correct prediction given aCP), These modules are linked by an associative learning network and an internal controller that ensures autonomous system operation in real time. The controller is designed to create the minimal number of AH.'Ta recognition categories, or "hidden units)), needed to meet accuracy criteria. It docs this by realizing a minimax learning rule that enables the fuzzy AitTMAP system to 2 learn quickly, efficiently, and accurately as it conjointly minimizes predictive error and maximizes predictive generalization. This scheme automatically links predictive success to category size on a trial-by-trial basis using only local operations. It works by increasing the vigilance parameter Pa of AHTa by the minimal amount needed to correct a predictive error at AR'l\. When the ART a classifier is presented with an input vector a, the bottom-up activation from Ft causes the F/j layer to choose a category node based on the input's fuzzy membership in that category's fur-zy set. T'he chosen category then sends information back to the Ff layer which is compared to the input vector a. The fuzzy intersection of top-down activation with the input vector produces a match value that indicates the classifier,s confidence in its category choice. Parameter Pa calibrates the minimum confidence that AH.Ta must have in a recognition category, or hypothesis, activated by an input aP in order for ART a to accept that category, rather than search for a better one through an automatically controlled process of hypothesis testing. Lower values of Pa enable larger categories to form leading to broader generalization and higher code compression. A predictive failure at ARTb increases Pa by the minimum amount needed to trigger hypothesis testing at AH..'I'a, using a mechanism called match tracking [5]. Match tracking sacrifices the rninimum amount of generalization necessary to correct a predictive error. Hypothesis testing leads to the selection of a new Alt.'I'a category, which focuses attention on a new cluster of aCP) input features that is better able to predict b(p). Fuzzy Altl'MAP can itself be used for multisensor fusion, by concatenating the information front all sensors into a single input vector. However, whenever a predictive error occurs during training, the match tracking signal resets the AH.T a classifier without regard to the predictive confidence in the individual channel information. Ill. FUSION AH.TMAP GENERALIZES FUZZY AHTMAP Fusion ARI'MAP extends the fuzzy ARTMAP classifier by incorporating an individual sensor classifier for each input channel, and by extending the match tracking technique in a manner that assigns blame only io the channels with least confidence in their predictions (Fig. 2). Before <'.t global recognition code is activated in